Post-production and image processing software applications can take advantage of custom hardware and, increasingly, of generic hardware in order to accelerate the processing of their video effects. Exploiting the processing speed-ups provided by newly introduced technologies becomes ever more important as the technical quality of cinematic and television content continues to improve. High definition home theatre systems with Blu-ray disc players have become a commodity, while digital cinema projection with 2K or even 4K line resolutions are gaining in popularity. This comes at the price of increased data processing requirements in the whole motion picture and broadcasting production pipeline. Image and video processing systems need to scale up their performance in order to handle such media.
Among the generic hardware available to media processing systems are various host CPU technologies, such as SSE2 for Intel-based platforms and Altivec for Apple Macintosh platforms, numerous graphics processing units (GPUs) commonly installed in off-the-shelf workstations, as well as other specialized hardware, such as Larrabee from Intel Corporation. The computer games market has helped make the GPU into a commodity. These GPUs have much more arithmetic processing power than equally priced central processing units (CPUs). GPUs make use of the inherent parallel nature of image rendering to outperform control-oriented CPUs, especially in gaming applications.
Image rendering for games and image processing applications involve similar processes. A developing research field, called general-purpose computing on GPU (GPGPU), explores techniques for using the GPU for a variety of problems including image processing. However, existing GPU-accelerated image processing systems are not easily programmed by developers using general purpose-programming languages such as C or C++. Current GPU-accelerated image processing systems require intricate knowledge of GPU components, the rendering pipeline, and rendering APIs.
There is also a general lack of GPU programming services that are directly targeted towards the needs of video processing on off-the-shelf platforms, operating systems, graphics cards and shader languages. Video processing requirements are primarily characterized by high bandwidth transfer requirements between host and GPU memory, and by other services that deal with the particular format and type of processing required for processing of video based visual effects.
The frequent introduction of new technologies greatly increases the complexity of software coding. In order to take advantage of the performance acceleration that the new technologies can provide, software needs to be intimately tied to the specific hardware type and model, the operating system, and the platform used. This means that in order to allow the same application to run optimally on different hardware and system configurations, multiple versions of the code are required.
The traditional approach to using newly introduced hardware is to develop individual low-level libraries specific to each new technology upon which the software is to run. These libraries are dedicated to and highly optimized for specific hardware and operating systems. They have their own protocol and particularities, and adopt a programming model that is often heavily influenced by the target hardware. This makes the application software difficult to maintain as the hardware evolves, and hinders rapid development of new software applications, such as new video effects. It prevents adoption of new hardware execution domains without the need for significant change in the client applications.
Once hardware-specific low-level libraries are developed, debugged and optimized, application developers and users seek to combine the strengths of these mature libraries. This is usually achieved by unifying some of the data structures and code of the various libraries. However, this can destabilize previously debugged software.